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In recent years, many fast-growing technologies coupled with wide volume of 
medical data for the digitalization of that data. Thus, researchers have shown 
their immense interest on Multi-sensor image fusion technologies which 
convey image information based on data from various sensor modalities into 
a single image. The image fusion technique is a widespread technique for the 
diagnosis of medical instrumentation and measurement. Therefore, in this 
paper we have introduced a novel multimodal sensor medical image fusion 
method based on type-2 fuzzy logic is proposed using Sugeno model. 
Moreover, a Gaussian smoothen filter is introduced to extract the detailed 
information of an image using sharp feature points. Type-2 fuzzy algorithm is 
used to achieve highly efficient feature points from both the b images to 
provide high visually classified resultant image. The experimental results 
demonstrate that the proposed method can achieve better performance than the 
state-of-the- art methods in terms of visual quality and objective evaluation. 
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1. INTRODUCTION 

In last few decades, huge amount of data have been generated by various healthcare systems in real 
time. Therefore, for increasing the quality of healthcare data, various healthcare systems are moving from 
hardcopy towards digital ones with cost minimization [1]. These much bulky data from various healthcare 
systems has a potential to support extensive range of healthcare and medical applications. Moreover, in order 
to treat patients efficiently they have clinical decision support which can cure patients precisely. Certainly, the 
most essential aspects for diagnosis of patients are the assessment of medical data available from patients and 
the decisions to be sanctioned. 

Recently, these bulky digital medical data have been associated with various rapid growing hi-tech 
innovations for the automation of digital data collection and storage and to develop more advanced techniques. 
Decision Support Systems (DSSs) helps the physicians in diagnosis of various patients by evaluating different 
type of diseases [2]. The use of DSS systems in association with numerous artificial intelligence techniques 
have helped for diagnosis and classify bulky and unmanageable amount of medical data which is gradually 
increasing all over world [3]. These type systems will help in evaluation of large medical data rapidly and more 
detailed manner [4]. 

Medical DSS more often obtain knowledge using real time data from various actual cases, whose 
training dataset (classification) is known. With the help of these training datasets, the novel models can be 
classified for the new incoming medical data. However, these decisions include various complex uncertainties 
related to medical diagnosis. Furthermore, different medical practitioners uses different decision making 
policies which may vary in nature. To reduce these type of complex model uncertainties and classification 
problems fuzzy logic has been widely adopt for several application fields. Specially, in medical applications 
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several DSS examples can be found which rely upon fuzzy logics [5] and can help in medical diagnosis of 
patients. 

Fuzzy logic is one among the most rapidly emerging technique in real time due to its effective 
handling capabilities in terms of uncertainty and demonstrating awareness in interpretable way. These can be 
attained with the help of approximate information and linguistic models using linguistic variables in numerical 
domain and correlations can be presented as logical guidelines. To develop fuzzy logics based DSSs, linguistic 
variables can be define to assemble huge informative knowledge, and fuzzy partitioning can be done to 
construct fuzzy rules. 

Sensory measurements and mathematical prototypes based Non-linear organized mapping can be 
achieved using fuzzy logics. Mainly, fuzzy logics can be classified into two types as type | and type 2 fuzzy 
logics. However, type-1 fuzzy logics consists of several issues such as lesser stability, noise and poor handling 
of uncertainties etc. Conventional type-1 fuzzy logic system (FLS) can be used in decision making of highly 
nonlinear system with limited range of uncertainty. To improve decision making capabilities of FLS’s 
presented in parameter uncertainties, adoption of type-2 FL is considered. Type-2 FL exhibit better capabilities 
in handling parameter uncertainties as it models vagueness and unreliability of information presented to it. 
Interval Type-2 fuzzy logic systems (IT2 FLSs) benefit from interval fuzzy membership functions which make 
them a preferable choice in the presence of uncertainties and/or appreciable level of noise in the system. 
Therefore, Type-2 fuzzy logics is preferred over type-1 of fuzzy logics. Type-2 sets and FLS’s have been used 
in decision making [6], [7] solving fuzzy relation equations [8], survey processing [8], time-series forecasting 
[9], [10], function approximation [8], control of mobile robots [11], biometrics [12], and preprocessing of data 
[13]. IT2 FLSs can cover a wide range of uncertainties, like different meanings of words, noisy input data, 
different decisions made by different experts, etc. 

In medical applications, fuzzy logics can be integrated with data fusion technique to fuse 
complementary information from two or more source images into a single image with more information and 
visually more attractive and can be process much efficiently. In various medical applications, both CT 
(Computed Tomography) and MRI (Magnetic Resonance Imaging) techniques disclose the structural and 
anatomical statistics, while both PET (Positron Emission Tomography) and SPECT (Single Photon Emission 
Computed Tomography) disclose the functional information. Therefore, multi-model medical image fusion 
technique can offer effective anatomical and physiological characteristics which will helps in image analysis, 
clinical diagnosis and effective treatment planning. 

Therefore, this paper demonstrate a novel image fusion technique based on interval type-2 fuzzy logic 
system (IT2FLS) to handle non-linear uncertainties and achieve unbiased medical diagnose in medical image 
datasets. In particular, Type-2 logic system helps to model type-2 membership functions using type-2 
membership variables. Here, type-2 fuzzy logics are used for higher degree stability and better management of 
uncertainty. The proposed fusion technique extract useful information from various CT, MRI, MR-T1 and MR- 
T2 images. First, we will take various combination of different input source images. Then, various patches 
extracted from both the images to detect essential feature points of both the images. These patches helps to 
extract image local features and helps to detect feature points which can integrated with type-2 fuzzy logics 
which evaluate weights for every image pixels present in various patches. In Next phase, the feature points are 
extracted. Feature points are used to determine the patterns of the different digital image as well as it helps to 
eliminate noise. It emphasizing pixels with gray tones and make them different than their neighbors. Feature 
point further passed to interval type-2 fuzzy logic system (IT2FLS). IT2FLS system used to determine an exact 
membership function, and also there is a measurement uncertainties. It also helps to set an enable modeling 
and minimizing the effects of uncertainties. Then using image fusion rule both images are fused together to 
integrate (complementary as well as redundant) information from multimodality in images to create a fused 
output image. It can be proved as significant technology for diagnostics and treatments in the field of medical 
instrumentation and measurement. It not only provides accurate description of the same object but also helps 
in required memory reduction by storing fused images instead of multiple source images. Simulation results 
shows that the superiority for our proposed scheme in terms of visual and quantitative analysis. 

This paper is organize in following sections, which are as follows. In section 2, we describe about the 
image fusion issues and how they can be eliminated by using our proposed model. In section 3, we described 
our proposed methodology for image fusion of various image classes. In section 4, experimental results and 
evaluation shown, and section 5 concludes our paper. 


2. LITERATURE SURVEY 

In recent years, a healthy amount of research has been carried out in the field of electronic technologies 
to enhance the performance of image fusion techniques. Image fusion techniques can widely use in medical 
applications, image processing applications, remote sensing, satellite imaging, face recognition and security 
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fields etc. To enhance further more image quality and visually more attractive fuzzy logics can be implemented 
with the use of data fusion technique in terms of fuse complementary information from two or more source 
images into a single image with more information. Type-2 sets and FLS’s have been used in decision making 
[6], [7] solving fuzzy relation equations [8], survey processing [8], time-series forecasting [9], [10], function 
approximation [8], control of mobile robots [11], biometrics [13], and preprocessing of data [12]. 

In [14], a type-2 fuzzy set introduced using a Karnik-Mendel Algorithm which is an iterative and 
computationally intensive type reduction algorithm. This model efficiently eliminates the numerical problems 
and prevent infinite loops. This program demonstrates the superiority of type-2 fuzzy sets. In [15], a conditional 
fuzzy set which is similar to type-2 fuzzy set is proposed based on the product of primary and secondary 
variables. Difference of both variables are independent to each other whereas in type fuzzy systems secondary 
variables rely upon the primary variables which increases the complexities of fuzzy type-2 and make difficult 
for analyzing. In [16], an image fusion technique used to fuse two source images. The fused image can be 
evaluates in the basis of weighted sum information. The degree of focus is evaluated for various input images 
using fuzzy inference system. Finally, the fused image is evaluated using degree of focus map which is 
computed with the use of weight. In [17], an image fusion technique is presented for medical images using two 
cascaded algorithm such as Non sub-sampled Contourlet transform (NSCT) domains and discrete wavelet 
transform (DWT). Here, MRI (magnetic resonance imaging) and CT (Computed Tomography) images are used 
for training of the model. First, a DWT technique is employed to get principal component and then fusion rule 
is applied for NSCT domain to increase contrast of the features. In [18], the fusion technique employed to get 
contextual information for an effective measurements aggregation and also a time series estimated technique 
presented to process the aggregated values. A Type-2 fuzzy logic system is used for the precise identification 
in events due to thier reasoning abilities under uncertainty. Also verified type 2 fuzzy systems superiority over 
type-1 fuzzy systems. In [19], a fuzzy clustering technique is used to improve the quality of noisy images by 
eliminating noise based on Wavelet and Bi-dimensional Empirical Mode Decomposition (BEMD). In this 
paper a segmentation technique also adopted to provide SNR rates using FCM clustering dataset. In [20], a 
multi-modal medical fusion technique is used which rely upon 2-D PCA. Here, a brain image dataset is used 
for the fusion implementation. Image fusion outcomes are compared with PCA based fusion technique in terms 
of seven quality assessment parameters. Proposed algorithm result outperform the current PCA technique. In 
[21], a SIST technique is presented to get low-pass and band pass sub-band images of similar size as source 
images. Then, a fusion based PCNN technique is used to fuse the low frequency and band pass sub-band 
coefficients. Here, PCNN technique used in SIST domain to enhance spatial frequency and lastly, fused image 
can be attained with the help of ISIST (Inverse SIST). 

In existing techniques, Noise and uncertainties of sensor data affect decision making capabilities. To 
negate effects of sensor data uncertainties on decision making, an interval type-2 based FLDS (IT2FLDS) 
system is introduced in their proposed paper for the fusion of medical images. Interval type-2 trapezoidal 
membership functions are used to handle uncertainties in input sensor data. Comprehensive details of type-2 
fuzzy logic is available in [22] and is used to model IT2FLDS proposed. Matlab/Simulink and interval type-2 
fuzzy logic system toolbox (IT2FLS) [23] is considered for building the IT2FLDS. Results obtained prove that 
type-2 FL based IT2FLDS system proposed handles sensor data uncertainties better compare to fuzzy type-1 
counterpart [24]. 


3. MEDICAL IMAGE FUSION REPRESENTATION USING FUZZY LOGICS 

Image fusion technique is used to retrieve complementary and redundant data from multiple source 
images to build a fused image [25]. This fused image provides detailed description of the same images and also 
provides accurate feature points. Fusion technique also reduces the storage capacities by loading fused image 
instead of multimodal input source images. In recent years, various image fusion techniques emerged for the 
medical images which usually consist pixel, feature and decision level fusion techniques. Image fusion can be 
widely used in various application such as in medical applications, image processing applications, remote 
sensing, satellite imaging, face recognition and security fields etc., However, existing fusion techniques 
consists of several issues such as high computational complexity, presence of uncertainties and noise, poor 
decision making and feature points of precision. 

Therefore, a fusion technique for medical images using Type-2 fuzzy logic is presented. The proposed 
algorithm collects maximal information from both source images and fuse into a single image which high 
visual quality. Type-2 fuzzy logics is used to extract local features and evaluate weights of each pixels. 
Simulation outcomes shows the superiority of our proposed scheme based on visual and a quantitative analysis. 
The most essential advantages of Type-2 fuzzy logics works efficiently even in presence of noise and allow 
better management of uncertainty. 
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Figure |. Block diagram of medical image fusion using type-2 fuzzy logic 


Figure 1 demonstrates the block diagram of medical image fusion using type-2 fuzzy logic. Our 
proposed image fusion model used to reduce noise in multimodal medical images and provide high quality 
visually attractive image. First, we will take various combinations of different input source images. Then, 
various patches extracted from both the images to detect essential feature points of both the images. In Next 
phase, feature points are extracted and further passed to interval type-2 fuzzy logic system (IT2FLS). The 
IT2FLS system used to determine an exact membership function, and there exists measurement uncertainties. 
It also helps to enable the modeling and minimizing the effects of uncertainties. Then an image fusion technique 
is applied to fuse source images to get redundant and complementary information in an output fused image. 
This technique can be very helpful for the diagnosis of medical images. It also reduces memory by fusing 
multiple number of source images into single one. 
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Figure 2. Architecture diagram of Interval type-2 fuzzy decision system (IT2FLDS) 


Here, Figure 2 represents the Architecture diagram of Interval type-2 fuzzy decision system for 
medical image fusion. It consists of several blocks which are described below. 
a. Crisp input 

Crisp input is defined as the inputs which straightly measured by sensors and forwarded to fuzzy logic 
systems. The inputs which is forwarded to fuzzy logic through sensors, consists its own set of memberships 
function. 
b. Interval Type-2 membership function 

Type-2 fuzzy set is expressed using fuzzy membership functions. In our medical image fusion model 
a Gaussian memberships function is used. Expert opinions are used to implement an interval type-2 Gaussian 
Fuzzy membership functions to eliminate uncertainties. Moreover, these type-2 fuzzy set considered as an 
input for fuzzy inference engine which is link with rule base. 
c. Fuzzifier 

Crisp inputs obtained through sensors which is faded into fuzzifiers and then it will transform into 
Interval Type-2 Fuzzy Decision System which rely upon fuzzy memberships function. This procedure is known 
as Fuzzification. Type-2 membership functions are linked with fuzzifiers to form an Input Processing Unit 
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which is demonstrated in Figure 1. This Input Processing Unit transform crisp inputs into type-2 fuzzy logic 
decision system. 
d. Rules 

Rules are described as “IF THEN “statements which is a group of linguistic variables. The linguistic 
variables linked with IF part termed as antecedent whereas the linguistic variables linked with THEN part is 
termed as consequent. AND operator can be utilized when model requires more than one rule. 
e. Inference 

Union and Intersection operators and rule base rules are utilized by Fuzzy Inference block to converts 
T2FLDS inputs into T2FLDS outputs. The Union and Intersection operators can be interchanged by Join (LI) 
and Meet operators (1) IN type-2 fuzzy logic based decision system. Secondary membership functions can 
utilize these two operators. 
f. Type-reduction 

TIFLS can be obtained from the reduced sets of Type-2 fuzzy outputs through fuzzy inference engine 
which can be termed as type-reduced sets. In IT2FLDS, there are two type of approaches present in the type- 
reduction such as Karnik Mendel iteration approach and the Wu-Mendel uncertainty bounds approach. These 
two approaches rely upon centroid which is used in our model. 
g. Defuzzification 

Type-reduce sets output forwarded to defuzzificaton block. Left end point and right end point are 
utilized to compute type-reduced sets. Average value of the points defines the defuzzified value. This 
defuzzification values further it converts to the output crisp sets where it takes the decision whether image 
fusion is accomplished or not. 

Initially, a medical image S is characterized in fuzzy domain with dimension P X Q with M levels is 
taken as a 2D array of fuzzy sets. 


S = {(spq.Upq(s))I(p,@) € [(0,0), (P — 1,Q -1)]} (1) 


Here, s,, represent the intensity value of every pixel and it consists of membership grade “,, which ranges 
from 0 to 1(0 S Ung S 1). The range of every membership function with respect to image spatial intensity 


value is [0,M — 1]. 


3.1. Gaussian based Interval Type-2 Fuzzy Membership Function 

Memberships function are used to characterize the properties of IT2FLDS sets. A medical image S 
with range [0,M — 1] can be transform into IT2FLDS property set which lies in the range of [0,1] using a 
Gaussian membership function which are expressed [26] as, 


~(5,—Spq)° /2F2 
U(Spq) = | (st Spq) /2F3| (2) 
Here, F,, represents a type-2 singleton fuzzifier whereas s; and s, represents max. and (p, q)th image 
scale values respectively. 
Here, to characterize spatial domain pixels in fuzzy domain we have presented a histogram based 
interval type 2 membership function which is expressed as, 


Un = el-(1-n)?/2F 2] (3) 


Where, n represents a definite values which lies in the range of [0, M — 1] and fuzzifier matrices F, 
can be represented as, 


FZ = Xnzo (Ss — 2)*f(n).2Can=o (r — 2)*f())* (4) 


Where, n denotes the intensity level and f(n) represents the frequency of occurrence of n intensity 
level in Histogram H, and f(n) can be expressed as, 


f@) =H, @.[(/-D@-DIr* (5) 


In a fuzzy domain, for a contrast-enhanced image, pixels can be of two types either dark (high) 
perception or light (low) perception which lies in the range from «, € [l,,1;] and w,, € [w,,u,] respectively. 
The pixels at « = 1, consist of highest ambiguity and do not lies in any of the perception. Let these pixels 
evaluates the fuzzy boundary then value at w = u; represent feature points. 
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3.2. Medical Image Feature Preservation: 

The Histogram Equalization Enlarging and Conversions techniques can be used to enhance the poor 
contrast of low quality images. In [27], an intensification operator is used to enhance contrast of the images 
which purely rely only upon membership function. The degradation of images are nonlinear and it performs 
uncertain activity in nature, thus, we introduce a modified non-linear contrast intensification operator 
MNINT[«,,] which consists of three tunable factors such as intensification operator I, fuzzier F, and crossover 
point s,. This contrast intensification function can be expressed as, 


-1 
uw (n) = MNINT[u(n)] = [4 + exp[-I(u(n) - >) || (6) 

Where, I used to define the shape of sigmoid function. The fuzzifier parameter F, and crossover 

point s, can be fine-tuned through (n) whereas intensification operator I remains fixed to control the contrast 


in improvement levels of the image. 


3.3. Fuzzy Type-2 Key Points Detection 
The modified contrast intensification operator MNINT[a,] can be restate in terms of each 


(p, q)th pixel, 
u'(p,q) = MNINT|tyq] = 1+ exp[—I(tyq — Sc) 7? (7) 


Here, we have presented a fuzzy parameterized Gaussian-type feature point detector which can be 
expressed as, 


6 
lu(@pt+x, qty)-u(p,q)] 
Ex Eyl - ) 





K(p,q) =e (8) 

Here, x,y € [—(m-— 1) /2,(m—1)/ 2], and the dimension of key feature point detector can be 
expressed asm X m. The center pixels can be expressed by membership value as w’(p,q) at the position 
of (p,q) whereas K (p,q) is the output feature point pixels which replaces the prior central pixels. Gaussian 
function consists of two adaptable fuzzifiers such as 6 and Fg. 

Hypothetically, factors 6 and Fg describes Gaussian function which are more expressive than usual 
factors such as @ and f as shown in [28]. A Gaussian membership function can be expressed as, 





| 
202 


(9) 


Ucauss(S; , C) = exp | 


The Gaussian curve rely upon the factors g and c.Whereas the factor 6 is expressed as the power of 
exponential term and a lone fuzzifier constant Fg can be achieved by decreasing the denominator of 
exponential term, 





-(s-c)\® 
Ugauss = exp ( 2p ) (10) 


=> 29= Fy (11) 


Different type of Gaussian functions can be derived by applying the various combination of 6 and Fe 
which are Gaussian types. This results are visually very poor with unsmooth and dense feature points when 
experimenting with various fuzzifiers 6 even values whereas odd integer values of fuzzifiers 5 gives thinner 
and refined feature points. When the low odd values are used as feature points which are visually not clear as 
well as weak whereas high odd values gives visually clear feature points and also eliminates weak 
feature points. 

Mathematically, the Gaussian function width can be expressed by the fuzzifier Fg. Thus, a thin 
Gaussian points can be achieved when low fuzzifer Fg value elected which can decreases the distance into a 
crisp set approximately. To obtain detailed information from the feature points, the function need to be closer 
to crisp set and reverse for the high fuzzifier Fg values. The factors 6 and F¢ need to be pre-determined to 
ensure decent performance in a particular application which can be achieved using preliminary experiments 
and entropy optimization. 
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3.4. Feature and Key Points Refinement 
Both strong feature points and impulse noise can be experienced when we have, 
1) Larger values of K,i.e.K (p,q) > 1; 
2) Lower values of K,i.e.K(p,q) <0. 
The feature points can be achieved between the ranges [0,1] approximately, when a feature point 
attenuator pattern applied. 


K'(p,q) =) [{K@, Dlp € [0,P — 1],q¢ € [0,Q—1]}], VK@,q) > 1; 
And (12) 


K'(p, q) =7 [{K(p, |p € [0,P — 1],q € [0,Q—-1]}], VK(p,q) <0 


Finally, a Binarized Key Feature point based image can be developed by applying a simple 
thresholding method. Various experiments evaluate that the optimal threshold level I should lie in the range 
of [YY], Gay 

* 1, or K'(p,q) => T 

K"(p,q) = { ‘i ia 


for K'(y,q) < T (13) 


Where, = [y,,y;],p € [0, P — 1] and q € [0, Q — 1].The use of Image thresholding step rely upon 
the application relevance. 


3.5. Optimization Adopted to Construct Weight Maps 

To achieve four tunable factors such as s,,F,,6 and Fg we require a fuzzy entropy optimization 
technique. The fuzziness degree of a set and image indefiniteness can be measured using entropy of a fuzzy 
set. Entropy /[29], can be expressed using Shannon’s function S;, which is as follows, 


I = (log, 2)~* Lin=o Sif(n) (14) 
In fuzzy, we can express S; as, 

S,(w'(n)) = —u'(n) log, u’(n) — (4 - u'(n)) log.(1 - u'(n)) (15) 
whereO < w' <1 

Entropy optimization technique can only optimize s, and F, out of three tunable factors globally. 


Optimization process is not necessarily required in predetermined factor I. The derivatives of J can be 
expressed in terms of s, and F,, 

















dl dl du'(n) x = , 
ha OO = (loge 21 DIP (wn) — 561 F@) (16) 
a _ ada du'(n) du(n) _ -10M-1 [12 u(n)(u(n)-se)(sp—n)?G(u’)] 
dF, du!(n)' du(n)" dF, (loge 2) n=0 | F3 ]£@) a7) 
Where G(uzz’)can be expressed as, 
—-I(u(n)-Se) 
G(w') = w'(n) (1-w'(n)) = — (18) 


[ate-Hun)-se)]* 


These derivatives can be used in recursive knowledge of the factors s, and F, by gradient descent scheme. 


dl 
Se = Se 15 (19) 


F, = F,- @ a (20) 


Where €, and €, are learning degrees for both factors s, and F, respectively. The adjacent 
optimization point in both positive and negative directions is used. 
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Optimization technique can also be applied on 6 and Fg to get its optimized values and as these factors 


are very much essential in extracting feature points. By considering Fuzzy entropy function locally, we can 
alter equation (15) by changing w'(n) to the local feature point pixel K (p,q), 


Si(K(@, q)) = —[K(p, q) loge K(p,q) + (1-— K(@, q)) loge(1 — Kp, q)) | (21) 


The derivatives of J in terms of 6 and F¢ can be expressed as, 








di dl_—aK(@,q) | _ yt Ny? Ne 
OF Tea a = KlogelK. (1 - K) 12,2, |(4) log. (=)| (22) 


di dI_—s dK(@q) __ K6XxLyNP {«) 
dFg  dK(pq)' dFg rott eu-K 








(23) 


Where N = [w’(p + x,q + y) — u'(s,h)]. Above derivatives can be used for recursively learning several 
factors using gradient descent scheme, 


ey esis 
6'=d-€, a (24) 
i dl 
Fe = Fe- €a55, (25) 
Where €3 and €, are the learning degrees for both factors 5 and Fg respectively. Here, we also applied gradient 
domain guided filtering wF, to evaluate every decision map such as D, ; D. ; and D3 ; which rely upon feature 
points with the respective source image S with its j*" component. Source image S is treated as a guidance image 


to produce the ultimate weight maps, 


Xf, = wFy1(D,;,5;) (26) 


Xfj = “F2(Di,,5)) (27) 
Where, b1 and b2 are the factors of gradient domain guided filtering and i = 1,2,3,..i,; and X represents 
weights to be evaluated. Here, X% represents the approximate value of weights and X@ represents the detailed 
value of weights. By combining all above calculations we can evaluate exact weight maps with respect to 


corresponding source image Sj with its j £2 component which is expressed as, 
Xf = Th Xf (28) 
d — Ty d 
XP = Ths Xj (29) 


Where, Xj and Xx? are resultant weight maps of low and high frequency components. Here, the L 
represents the number of source images and finally, to sum every pixels into one all the values of L image 
weights maps are normalized. 


3.6. Medical Image Fusion Considering Feature and Weight Maps: 


The weighted averaging technique can be used to fuse approximate and detailed components of 
various source, 


Sf = Li XPS? (30) 


sd = DExesa G1) 


In last, the fused image F can be represented in terms of fused approximate Sf and detailed 
components S¢ as, 


F = Sit S¢ (32) 
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4. PERFORMANCE EVALUATION: 

To determine the performance of our proposed image fusion model using type-2 fuzzy logics decision 
system T2FLDS five different set of medical images is considered. Matlab and Simulink are mainly used to 
carry out all the experiment. Type-2 of fuzzy toolbox [30], [31] is used to implement IT2FLDS. The 
performance of the image fusion model is determined using suitable fuzzy rules, membership function and 
Fuzzy Inference Engine (FIE).The experimental outcome clears that the IT2FLDS can handle better 
uncertainties in contrast to type-1 FLDS. Moreover, the primary experiment are further extended for handling 
the medical image fusion processing through the sensor data. The proposed image fusion model is implemented 
on clinically acquired various multi-modal medical images. The medical image dataset used in our experiments 
are taken from http://www.imagefusion.org/ and http://www.med.harvard.edu /aanlib/home.html which is an 
open source. The performance of proposed method can be evaluated based on the experiments carried on 
various medical images with different modalities. 


4.1. Evaluation Criteria 

This section describes the objective analysis of our proposed model using certain quantitative 
measures. There are various image quality measures to determine the visual quality of medical images in 
various aspects. Here, in this paper, we have considered both quantitative analysis and visual representation of 
the fused medical images. To evaluate the performance of proposed medical image fusion technique we have 
taken three distinct fusion performance metrics which are expressed as, 


4.1.1. Feature Point Similarity Measure (Q42//) 
To evaluate the similarity between the feature points which are moved from base images to the fused 
image we have used Gradient based index (Q**/*) [32] quality assessment matric which can be expressed as: 


QX/F = we dda? (ab) g*(ab)+Q"? (ab) g* (ab) 


xP o elo ab) +g" @b)] G2) 

Where, 
QV" (a,b) = QV" (a, b)Q/"* (a, b) (34) 
Q’* (a, b) = Q?* (a, b)Q’* (a, b) (35) 


Where, the gradient strengths of two different medical images X and Y can be presented as g*(a, b) 
and g* (a,b), respectively. The image width and image height can be expressed as P and Q whereas 
Qx* (a, b) and Q** (a, b) represents the strength of feature points for image X and Y respectively and for every 
base image orientation preservation values at place (a, b).To evaluate the quantity of gradient information 
inserted into the fused image from the base images we have used a popular fusion metric Q*¥/*, 


4.1.2. Mutual Information 

To specify amount of information of the fused image which conveys information about the base 
image, we present a Mutual information (MI) [33] quality metric. The mutual information between the fused 
and base image can be expressed as, 


MI = MIX* + MIX? (36) 
Where, 
XF yh yh Xr WY ow) 
MIP? = Yo Dh g HX* (c,w) log, (=) (37) 
HY (h, ) 
MIM? = Yo Dh -o HY” (hh, w) log, (=) (38) 


Where, the normalized mutual information between fused and base images X and Y can be expressed 
by MI** and MI"™, respectively where c,h, and w € [0,L]. The normalized histograms of the base and fused 
image can be denoted as H*(c),H* (Ih)), and H¥ (w).The joint histograms between the fused and base images 
X and Y can be denoted as H**(c, w) and H” (lh, w). High MII value shows the superiority of proposed 
fusion model. 





Fine grained medical image fusion using type-2 fuzzy logic (Ramya H R) 


1008 O ISSN: 2502-4752 


4.1.3. Standard Deviation (STD) 

To evaluate the amount of widening of grey values in a medical image, here we introduce a standard 
deviation metric. More the standard deviation, better the fusion results. The Standard Deviation (STD) can be 
expressed as, 


1/2 
STD = (“Sh Dib) - 0) (39) 


Where, F(a, b) represents the pixel values of fused image at the location (a,b) and ji represents the mean 
value of that image. 


4.2. Experiments and Discussions 

We implemented a comprehensive objective analysis on the experimental outcomes by our proposed 
image fusion technique with Type-2 fuzzy logics. Here, Table 1 represents the quantitative performance 
comparisons of proposed technique in contrast with the conventional state-of-art-techniques for five different 
group of medical images. In Table 1, three quality analysis matrices are presented in which MI and STD are 
largest objective matrices and Q**/* lies in the range of 0 to 1. The larger MII value shows the superiority of 
our proposed fusion technique. Table 1 quantitative outcomes shows the robustness of our proposed technique 
in terms of spatial consistency as well as significant extracted information of our proposed method performs 
better than the conventional state-of-art fusion techniques. The better results of quality assessment matrices 
represents that the fused image carried out using our proposed technique can hold much more detailed 
information can attain a larger similarity and correlation to the base image when compared to other state-of-art 
fusion techniques. 

We have performed our experiments on different multimodal medical image over the standard medical 
test images to validate the feasibility of our proposed image fusion technique. There are five different medical 
image group of each type are available and the two base images are perfectly pre-registered for every group 
pair as presented in Figure 3. 

The proposed image fusion technique compared to seven other state-of-the-art image fusion 
techniques which rely upon the methods like GP, DWT, CT, NSCT, Scheme-1 [34], Scheme-2 [35] and NSCT 
+ T2FLDS methods as presented in Figure 4. The proposed image fusion model with IT2FLDS technique 
outperforms all the other state-of-art-techniques in terms of Mutual Information (MI), Standard Deviation 
(STD) and Feature Point Similarity Measure (Q*¥/*) as presented in Table 1. The visual quality of medical 
images is much better using our proposed model in contrast to other state-of-art-techniques. 


Table 1. Evaluation Indices for Fused Medical Images 








Group Indices GP DWT DTCWT CT NSCT Scheme- Scheme- NSCT PROPOSED 
1[34] 2[35] T2FLDS 
Group] MI 2.9816 2.3709 2.3663 2.1769 2.5573 2.6506 4.3924 4.5619 5.7741 


CT/MRI — AB/F 0.652 0.618 0.6464 0.5763 0.6913 0.7014 0.7889 0.7859 0.6218 
STD 38.782 41.159 39.791 40.318 41.248 59.188 58.257 60.871 92.50 


Group2 MI 3.3544 3.1367 3.2059 3.1086 3.2943 3.2339 3.0614 3.9635 3.837508 
MR- AB/F 0.5867 0.567 0.5998 0.5541 0.6287 0.5942 0.5802 0.6412 0.585129 
TI/MRA_ STD 68.3325 72.2603 70.9012 71.3378 72.219 82.6464 72.1308 90.1086 90.48137 
Group3 MI 3.8479 3.1827 3.3156 3.1689 3.4469 2.8063 4.5088 4.6146 6.782690 


CT/MRI AB/F 0.5665 0.5299 0.5631 0.5297 0.5926 0.5206 0.6268 0.6469 0.656692 
STD 46.2356 50.8396 50.067 50.4575 50.9719 57.601 69.1269 69.1941  68.47939 





Group4 MI 3.5102 3.2095 3.298 3.1454 3.4207 3.5195 3.5107 4.0579 4.294069 
MR- AB/F 0.6632 0.6347 0.6742 0.6333 0.6862 0.6862 0.6641 0.7154 0.677208 
TI/T2 STD 34.585 37.3586 36.9695 37.3167 = 37.37 40.4041 38.4298 43.2922 53.3414 

Group5 MI 3.3656 3.1878 3.2148 3.1445 3.2716 3.3301 3.3819 3.6634 5.086480 
MR- AB/F 0.5579 0.5266 0.5692 0.519 0.5848 0.5755 0.5631 0.6197 0.621281 


GAD/T2 STD 62.5626 65.6693 64.7158 65.1074 65.4498 68.8262 70.1364 80.0055 — 85.94753 





5. CONCLUSION 

The growth of information processing technologies and frequent development of medical imaging 
have given rise to challenge of clinical assessment of medical images. Thus, Multimodal medical image fusion 
technique can prove a vital asset for the various clinical imaging sensor applications. Therefore, in this paper, 
a novel multimodal sensor medical image fusion technique based on type-2 fuzzy logics is presented. 

To enhance the visual quality of reconstructed medical images and eliminate the drawbacks of existing 
state-of-art-techniques a Type-2 fuzzy logics based on Sugeno model is presented to carried out more detailed 
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information, comprehensive objective analysis, spatial consistency, attain larger similarity and correlation. 
Experimental outcomes clearly demonstrates that our proposed technique can provide detailed fused image 
information and also enhance visual quality of medical images and increases similarity between base and fused 
image using type-2 Fuzzy logics. Moreover, our experimental results outperforms all the other existing state- 
of-art-techniques in terms of mutual information, standard deviation and Feature Point Similarity Measure 
which highlights the robustness and high efficiency of our proposed method. In future work, we will 
demonstrate our results on PET/MRI and SPECT/MRI colored images and compared with other state-of- 
art methods. 





Figure 3. Source multimodal medical images: (al), (b1) image group 1 (CT and MRI); (a2), (b2) image group 
2 (CT and MRI); (a3), (b3) image group 3 (MR-T1 and MRA); (a4), (b4) image group 4 (MR-T1 and MR- 
T2); (a5), (b5) image group 5 (MR-Gad and MR-T2) 
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(A2) (B2) (C2) (D2) (E2) (F2) (G2) (H2) (12) 





(B3) (C3) (D3) (E3) (F3) (G3) (H3) (13) 





(B4) (C4) (D4) (E4) (F4) (G4) (4) (14) 





(B5) (C5) (D5) (E5) (F5) (G5) (H5) (5) 


Figure 4. Visual results for the five pairs of CT/MRI images. Fused images from (al)—(a5) GP based method; 
(b1)-(b5) DWT based method; (cl)—(c5) DICWT based method; (d1)—(d5) CT based method; (e1)—(e5) 
NSCT based method; (f1)—(f5) scheme [38] method; (g1)-(g5) scheme [40] method; (h1)-(h5)NSCT + 

T2FL; (11) - (15)proposed method 
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